youtube
This course is a guide to understanding and implementing Llama 4. @vukrosic will teach you how to code Llama 4 from scratch.
Code and presentations:
Code DeepSeek V3 From Scratch:
⭐️ Contents ⭐️
- 0:00:00 Introduction to the course
- 0:00:15 Llama 4 Overview and Ranking
- 0:00:26 Course Prerequisites
- 0:00:43 Course Approach for Beginners
- 0:01:27 Why Code Llama from Scratch?
- 0:02:20 Understanding LLMs and Text Generation
- 0:03:11 How LLMs Predict the Next Word
- 0:04:13 Probability Distribution of Next Words
- 0:05:11 The Role of Data in Prediction
- 0:05:51 Probability Distribution and Word Prediction
- 0:08:01 Sampling Techniques
- 0:08:22 Greedy Sampling
- 0:09:09 Random Sampling
- 0:09:52 Top K Sampling
- 0:11:02 Temperature Sampling for Controlling Randomness
- 0:12:56 What are Tokens?
- 0:13:52 Tokenization Example: "Hello world"
- 0:14:30 How LLMs Learn Semantic Meaning
- 0:15:23 Token Relationships and Context
- 0:17:17 The Concept of Embeddings
- 0:21:37 Tokenization Challenges
- 0:22:15 Large Vocabulary Size
- 0:23:28 Handling Misspellings and New Words
- 0:28:42 Introducing Subword Tokens
- 0:30:16 Byte Pair Encoding (BPE) Overview
- 0:34:11 Understanding Vector Embeddings
- 0:36:59 Visualizing Embeddings
- 0:40:50 The Embedding Layer
- 0:45:31 Token Indexing and Swapping Embeddings
- 0:48:10 Coding Your Own Tokenizer
- 0:49:41 Implementing Byte Pair Encoding
- 0:52:13 Initializing Vocabulary and Pre-tokenization
- 0:55:1
|
2025/04/24
youtube
